Many distributed systems utilize static networks, wherein processing being performed by one network node is not easily transferrable to another node without losing progress. For example, in a neural network, training progress for a node and/or for an entire network may be lost when a single node of the neural network fails.